Decision Tree Induction Systems: A Bayesian Analysis

Decision tree induction systems are being used for knowledge acquisition. Yet they have been developed without proper regard for the subjective Bayesian theory of inductive inference. This paper examines the problem tackled by these systems from the Bayesian view in order to interpret the systems and the heuristic methods they use. It is shown that decision tree systems depart from the usual Bayesian methods by implicitly incorporating prior belief that the simpler of two hypotheses will be preferred, all else being equal. They perform a greedy search of the space of rules to find one in which there is strong posterior belief.